In many situations it is desirable to capture an entire large scene and view the captured scene as a wide-angle image. Wide-angle cameras typically are used to capture these large scenes and provide the wide-angle image of the scene. In general, a wide-angle camera has a field-of-view greater than approximately 60 degrees. Additionally, a wide-angle camera includes a panoramic camera, which is an extremely wide-angle camera having a field-of-view of up to 360-degrees. Wide-angle views can be recorded with current systems either with special lenses or by sweeping a camera across a scene. The former method suffers from low and/or irregular resolution. The latter method cannot record a scene all at one time.
A wide-angle camera also enables instantaneous virtual panning and zooming. A wide-angle camera captures an entire or large portion of a scene in a single image, and therefore panning and zooming can be performed instantaneously without physically moving the camera or camera lens. In addition, a wide-angle camera can be used to provide guidance for a secondary pan/tilt/zoom camera. Thus, wide-angle cameras are highly desirable in many situations.
Under normal viewing conditions, the wide-angle images obtained from wide-angle cameras suffer from two kinds of distortions. First, normal perspective images can create distortions especially near the corners. Second, the apparent depth differences between objects are often exaggerated.
Wide-angle images obtained from wide-angle cameras can be viewed using at least two types of projections. A linear perspective projection is an image captured by a wide-angle lens that is projected onto a flat piece of film. Linear perspective projection keeps straight lines straight at the expense of maintaining shape. This causes perspective deformations. The deformations on the image plane make the user feel that the scene is not rigid, as if the scene were swimming around the viewer, particularly when viewing panoramic images.
A cylindrical projection is produced using a wide-angle camera having curved film and a rotating slit lens. Cylindrical projections are better at maintaining shape than linear projections. However, the cylindrical projection curves straight lines more than necessary.
A related problem is that wide-angle images exaggerate the depth disparity between near and far objects. One important visual cue of the depth of an object in a scene is the ratio between the image sizes of similar objects placed at near and far locations (called the depth foreshortening ratio). A smaller field-of-view results in a smaller foreshortening ratio. When a wide-angle image of a deep scene (such as a video conferencing scene) is viewed on a computer monitor, the viewer's field-of-view, in general, is much smaller than the field-of-view of the actual images. Therefore, the depth perceived by the viewer is much larger than the actual depth.
One application where these depth (along with other distortion and perception) problems and errors manifest themselves is in video conferencing applications. Wide-angle cameras often are used in video conferencing systems to capture and transmit a wide-angle image containing all participants present in a meeting room. When viewing the videoconference, however, the wide-angle image exaggerates the depth of the room. This causes the people in the middle of the image (who are usually furthest away from the camera) to appear quite small compared to others in the room due to the extreme foreshortening.
Video conferencing systems often employ a pan/tilt/zoom camera that has a motor that allows the camera to turn to find the current speaker and then zoom in for a closer look. This type of system has latency problems since it must first turn and zoom to find a new subject of interest.
Other video conferencing systems use a wide-angle camera having a plurality of cameras. By covering the whole room simultaneously, the latency problem is overcome. Typically, each of the plurality of cameras has the same field-of-view. This means that a wide-angle image captured by the wide-angle camera has a uniform pixel density across the entire wide-angle image. Pixel density is defined as the number of pixels per area of the image. Typically, the captured wide-angle image is processed to attempt to alleviate distortion and perception problems and errors. Using the example above, a captured wide-angle image may contain a scene of participants in a conference room. In order to better view the meeting participants (which are very small) in the center of the wide-angle image, one could uniformly zoom up a sub-region in the center and show the center portion only. Another possibility, as will be discussed below, is to scale the image differently at different locations so that the center portion is scaled up more while still keeping all the people in the image. The center of the wide-angle image can be enlarged to make the furthest participant about the same size as the other meeting participants at the outer portions of the wide-angle image.
One problem, however, is that the pixel density is small in the center of the wide-angle image. By way of example, three or four pixels may be all that represent a meeting participant's face at the center of the wide-angle image. When the center image is enlarged, the three or four representative pixels become blobs. This makes it difficult if not impossible to effectively eliminate distortion and perception problems and errors in the wide-angle image.